The following content has been provided by the University of Erlangen-Nürnberg.

Okay, so let's start. Professor Hornegis is not here today and he asked me to do the lecture,

so let me first introduce myself. I'm Jakob Wasse. I'm at the Pattern Recognition Lab and I'm working in the Medical Image Processing Group,

more precisely, Medical Image Registration Group. And today's topic is about image enhancement, which is a pre-processing technique.

And here's a short outline of the lecture today. I'll start with a motivation, try to put this lecture or one of our research fields

in the context of this lecture, Interventional Medical Image Processing, and its research fields are so-called range imaging sensors.

And here we'll motivate this with range imaging in abdominal surgery. The main part of this lecture today will be, of course,

the actual techniques that we're using here. Give us some notes about the concept of normalized convolution bilateral filtering

as an edge preserving denoising technique and recently proposed concept of guided filtering, which we go into very detail.

At the end, I'll give you short notes on real-time pre-processing using GPUs. These are hardware acceleration strategies.

And here we'll just briefly tell you something about how you can actually implement these methods in the GPU and some numerical issues

that arise from a porting algorithm to this graphics processing. So first of all, motivation range imaging sensors.

Do you know what range imaging sensors are? Do you know what these devices here? You know perhaps this device, Microsoft Kinect.

Originally developed for consumer electronics where you have control of free gaming, yeah, but the sensor has, it basically does nothing else

but it captures the scene. You have a depth image, metric surface information, and for example, the system can then segment the user

and try to recognize gestures or something like that so you have control of free gaming. Another device that we're using here is

ChemCube. This is a time of flight sensor so you can see also from the principle they rely on different techniques.

And here's some basic facts like resolution, frame rate, and field of view or noise level.

Interesting here is these devices here cost approximately 8,000 euros, so they're very expensive in contrast here.

A consumer device which is about 100 euros, so it's a really cool device also for doing serious research, right?

What we're doing, or one field of research that we're doing at the Pattern Recognition Lab with these devices are abdominal surgery techniques.

And here, the basic idea for example is that we have preoperatively acquired CT data and we want to fuse this preoperatively acquired data

with intraoperative surface measurements that we obtain from these range imaging devices.

And one application for example would be a liver resection and the liver is an organ that has very much blood vessels in it.

So if the surgeon's doing the wrong cut, well the patient tends to die. So this is a very serious injury that can happen here.

And what we want to do is that we want to augment the physician's field of view with this preoperatively acquired data.

And we do this by registration techniques tomorrow. You also have a guest lecturer and you'll learn more about that, how you can actually register these two modalities.

Here this is an example from open surgery. Another part we're doing, or another field we're doing is endoscopy.

And the common endoscopes they only have 2D RGB data, so you have no depth information.

And what we're doing at the lab is that we fuse this conventional 2D RGB data with range images or depth information.

And possible applications for this are that you can automatically measure regions of interest or organs or polyps or whatever you want.

You can also segment and track tools because the tools are in general very sharp objects, right?

So they may harm the organ, organs or the belly or whatever you want.

You can also do navigation and collision avoidance because you easily lose track if you have no depth perception with these endoscopes.

So the basic problem, however, that we have with all these range imaging devices is the raw data that we obtain from the sensor here on the left.

We have a very low signal to noise ratio, or in other words the data is very noisy.

We have artifacts here and preprocessing is here really a fundamental step that you can do registration, algorithmic segmentation or whatever you want.

So the first step that we usually do is that we apply temporal averaging. This is a very simple technique.

I won't go into this in this lecture today. However, a really important part is the last step here and this is bilateral filtering.

This is an edge preserving denoising technique. So for example you can see here on this plateaus here that are still very noisy.

And here we have a very smooth surface while we retain these sharp edges here. So edge preserving denoising is very important.

I also have a video for you where you can see actual live sequence that we took.

You can see here these outliers that are due to specular reflections and these specular reflections they typically occur with these imaging devices on organ surfaces due to a very wet and shiny surface.

You see we can eliminate them as a first step then we apply temporal averaging so we get a much more stable or steady surface. However, it's still rough.

And in the last step we actually apply edge preserving denoising technique and you can see here that we get a smooth surface while we retain the actual shape of the object.

So far for the motivation. So now let's go into detail of these techniques that we're using.

First of all I'd like to introduce the notation that I'm using today. Perhaps Professor Hornig is using kind of different notation. However, I'm sure you can follow me anyway.

So we consider always discrete images in this lecture today. So in general you have the problem that you have in continuous domain especially later in this term you will know or learn about variational calculus methods.

And there it is really important that you formulate all your problems in the continuous domain. So speaking here discrete images we have sums.

So in a continuous domain you simply replace the sums with an integer. So the number of pixels that we have simply n and one index can be accessed by this vector here.